Vector Equations
Contents
Vector Equations#
Vectors#
We’re going to expand our view of what a linear system can represent.
Instead of thinking of it as a collection of equations, we are going to think about it as a single equation.
This is a major shift in perspective that will open up an entirely new way of thinking about matrices.
(…and it’s going to open the door to computer graphics, machine learning, and statistics … later on!)
To make this fundamental shift we need to introduce the idea of a vector. We’ll mainly talk about vectors today.
We’ll then return to thinking about a linear system – now interpreted as a vector equation – in the next lecture.
A matrix with only one column is called a column vector, or simply a vector.
Here are some examples.
These are vectors in \(\mathbb{R}^2\):
These are vectors in \(\mathbb{R}^3\):
There is an important notational point here:
When we write \(\mathbb{R}^n\), we mean all the vectors that have exactly \(n\) components.
Vectors are Fundamental Objects#
We are going to define operations over vectors, so that we can write equations in terms of vectors.
In particular, we now define how to compare vectors, add vectors, and multiply vectors by a scalar.
First: Two vectors are equal if and only if their corresponding entries are equal.
Thus \(\left[\begin{array}{c}7\\4\end{array}\right]\) and \(\left[\begin{array}{c}4\\7\end{array}\right]\) are not equal.
Next: Multiplying a vector by a scalar is accomplished by multiplying each entry by the scalar.
For example:
And finally: the sum of two vectors is the vector whose entries are the corresponding sums.
For example:
Note that vectors of different sizes cannot be compared or added.
For example, if \({\bf u} \in \mathbb{R}^2\) and \({\bf v} \in \mathbb{R}^3\):
we cannot ask whether \({\bf u} = {\bf v}\), and
\({\bf u} + {\bf v}\) is undefined.
So with these three definitions, we have all the tools to write equations using vectors.
For example, if \({\bf u} = \left[\begin{array}{r}1\\-2\end{array}\right]\) and \({\bf v} = \left[\begin{array}{r}2\\-5\end{array}\right]\) then
Vectors Correspond to Points#
As already noted, an ordered sequence of \(n\) numbers can be thought of as a point in \(\mathbb{R}^n\).
Hence, a vector like \(\left[\begin{array}{c}-2\\-1\end{array}\right]\) can be thought of as a point on the plane.
We will sometimes write vectors like this: \((-2,-1)\) but this is not the standard form.
Sometimes we draw an arrow from the origin to the point.
This style comes from physics, but can be a helpful visualization in any case.
Vector Addition, Geometrically#
A geometric interpretation of vector sum is as a parallelogram.
If \({\bf u}\) and \({\bf v}\) in \(\mathbb{R}^2\) are represented as points in the plane, then \({\bf u} + {\bf v}\) corresponds to the fourth vertex of the parallelogram whose other vertices are \({\bf u}, 0,\) and \({\bf v}\).
This should be clear from the definition of vector addition (i.e., addition of corresponding elements).
Vector Scaling, Geometrically#
For a given vector \({\bf v}\) and a scalar \(a\), multiplying \(a\) and \({\bf v}\) corresponds to lengthening \(\bf v\) by a factor of \(a\).
So \(2\bf v\) is twice as long as \(\bf v\).
Multiplying by a negative value reverses the “direction” of \(\bf v\).
The Algebra of \(\mathbb{R}^n\)#
We’ve defined a new object (the vector), and it has certain algebraic properties.
\({\bf u} + {\bf v} = {\bf v} + {\bf u}\)
\(({\bf u} + {\bf v}) + {\bf w} = {\bf u} + ({\bf v} + {\bf w})\)
\({\bf u} + {\bf 0} = {\bf 0} + {\bf u} = {\bf u}\)
\({\bf u} + ({\bf -u}) = {\bf -u} + {\bf u} = {\bf 0}\)
\(c({\bf u} + {\bf v}) = c{\bf u} + c{\bf v}\)
\((c+d){\bf u} = c{\bf u} + d{\bf u}\)
\(c(d{\bf u}) = (cd){\bf u}\)
\(1{\bf u} = {\bf u}\)
You can verify each of these by from the definitions of vector addition and scalar-vector multiplication.
Linear Combinations#
A fundamental thing we will do is to construct linear combinations of vectors:
The \(c_i\) values are called weights. Weights can be any real number, including zero. So some examples of linear combinations of \(\mathbf{v_1}\) and \(\mathbf{v_2}\) are:
and
Linear Combinations, Geometrically#
A Fundamental Question#
We are now going to take up a very basic question that will lead us to a deeper understanding of linear systems.
Given some set of vectors \({\bf a_1, a_2, ..., a_k}\), can a given vector \(\bf b\) be written as a linear combination of \({\bf a_1, a_2, ..., a_k}\)?
Let’s take a specific example.
Let \({\bf a_1} = \left[\begin{array}{c}1\\-2\\-5\end{array}\right], {\bf a_2} = \left[\begin{array}{c}2\\5\\6\end{array}\right],\) and \({\bf b} = \left[\begin{array}{c}7\\4\\-3\end{array}\right]\).
We need to determine whether \({\bf b}\) can be generated as a linear combination of \({\bf a_1}\) and \({\bf a_2}\). That is, we seek to find whether weights \(x_1\) and \(x_2\) exist such that
Solution. We are going to convert from a single vector equation to a set of linear equations, that is, a linear system.
We start by writing the form of the solution, if it exists:
Written out, this is:
By the definition of scalar-vector multiplication, this is:
By the definition of vector addition, this is:
By the definition of vector equality, this is:
We know how to solve this! Firing up Gaussian Elimination, we first construct the augmented matrix of this system, and then find its reduced row echelon form:
Voila!
Now, reading off the answer, we have \(x_1 = 3\), \(x_2 = 2\). So we have found the solution to our original problem:
In other words, we have found that
A Vector Equation is a Linear System!#
And vice versa!
Let’s state this formally. First, of all, recalling that vectors are columns, we can write the augmented matrix for the linear system in a very simple way.
For the vector equation
the corresponding linear system has augmented matrix:
Then we can make the following statement:
A vector equation
has the same solution set as the linear system whose augmented matrix is
This is a powerful concept; we have related
a single equation involving columns
to a set of equations corresponding to rows.
Span#
If a vector equation is equivalent to a linear system, then it must be possible for a vector equation to be inconsistent as well.
How can we understand what it means – in terms of vectors – for a vector equation to be inconsistent?
The answer involves a new concept: the span of a set of vectors.
Let’s say we are given vectors \({\bf a_1}, {\bf a_2},\) and \({\bf b}\).
And, say we know that it is possible to express \(\mathbf{b}\) as a linear combination of \(\mathbf{a}_1\) and \(\mathbf{a}_2\).
That is, there are some \(x_1, x_2\) such that \(x_1{\bf a_1} + x_2{\bf a_2} = {\bf b}.\)
Then we say that \({\bf b}\) is in the Span of the set of vectors \(\{{\bf a_1}, {\bf a_2}\}.\)
More generally, let’s say we are given a set of vectors \({\bf v_1, ..., v_p}\) where each \({\bf v_i} \in \mathbb{R}^n.\)
Then the set of all linear combinations of \({\bf v_1, ..., v_p}\) is denoted by
and is called the subset of \(\mathbb{R}^n\) spanned by \({\bf v_1, ..., v_p}.\)
Span of a single vector in \(\mathbb{R}^3\)
Span of two vectors in \(\mathbb{R}^3\)
Asking Whether a Vector Lies Within a Span#
Asking whether a vector \({\bf b}\) is in Span\(\{{\bf v_1, ..., v_p}\}\) is the same as asking whether the vector equation
has a solution.
… which we now know is the same as asking whether the linear system with augmented matrix
has a solution.
Let \({\bf a_1} = \left[\begin{array}{c}1\\-2\\3\end{array}\right], {\bf a_2} = \left[\begin{array}{c}5\\-13\\-3\end{array}\right],\) and \({\bf b} = \left[\begin{array}{c}6\\8\\-5\end{array}\right]\).
Then Span{\(\bf a_1, a_2\)} is a plane through the origin in \(\mathbb{R}^3\). Is \(\bf b\) in that plane?
Solution: Does the equation \(x_1{\bf a_1} + x_2{\bf a_2} = \bf b\) have a solution?
To answer this, consider the equivalent linear system. Solve the system by row reducing the augmented matrix
The third row shows that the system has no solution.
This means that the vector equation \(x_1{\bf a_1} + x_2{\bf a_2} = \bf b\) has no solution.
So \({\bf b}\) is not in Span{\(\bf a_1, a_2\)}.
What does this situation look like geometrically?
Two Distinct Vector Spaces#
Be sure to keep clear in your mind that we have been working with two different vector spaces.
One vector space is for visualizing equations that correspond to rows.
The other vector space is for visualizing vector equations (involving columns).
These are two different ways of visualizing the same linear system.
Let’s look at an inconsistent system both ways:
Here are the views, first as a vector equation, and then as a system of equations.
The second vector space (and visualization) is more familiar to us right now.
However, as the course goes on we will use the first vector space – the one in which columns are vectors – much more often. Make sure you understand the figure!